Skip to content

Fix XCom key handling when keys contain special characters like /#56042

Closed
Brunda10 wants to merge 5 commits intoapache:mainfrom
Brunda10:Xcom_Slash_key-Fix
Closed

Fix XCom key handling when keys contain special characters like /#56042
Brunda10 wants to merge 5 commits intoapache:mainfrom
Brunda10:Xcom_Slash_key-Fix

Conversation

@Brunda10
Copy link
Contributor

@Brunda10 Brunda10 commented Sep 24, 2025 •
edited
Loading

Closes : #55410
Summary

This PR fixes an issue where XCom keys containing special characters (for example /) would break API calls and cause 404 Not Found errors.

The problem happened because keys were passed directly in the URL without encoding, so FastAPI treated / as a path separator.

Changes Made

SDK (client.py, taskrunner.py)

  • Keys are now URL-encoded (quoted) before sending them to the API.

APIs (core_api/routes/public/xcoms.py, execution_api/routes/xcoms.py)

  • Keys from the URL are now decoded (unquoted) before looking them up in the database.

This PR ensures:

  • Keys are always quoted when sent (safe for URLs).

  • Keys are always unquoted when received (correct DB lookup).

  • Aligns XCom behavior across different access points (UI, API, execution API, and task runner).

Testing:

from airflow.decorators import dag, task

@dag(schedule=None)
def xcom_slash():

    @task
    def this_works() -> dict[str, str]:
        return {"Some Key": "Some Value"}
    @task
    def this_works_as_well() -> str:
        return "/some/path/like/key"

    @task
    def this_does_not_work() -> dict[str, str]:
        return {"key with slash /": "Some Value"}

    this_works() >> this_works_as_well() >> this_does_not_work()

dag = xcom_slash()
Screenshot from 2025-09-24 17-16-17

@boring-cyborg boring-cyborg bot added area:API Airflow's REST/HTTP API area:task-sdk labels Sep 24, 2025
@Brunda10 Brunda10 changed the title Xcom slash key fix Fix XCom key handling when keys contain special characters like / Sep 24, 2025
ashb
ashb reviewed Sep 24, 2025
View reviewed changes
Copy link
Member

@ashb ashb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Needs unit tests adding please

pierrejeambrun
pierrejeambrun reviewed Sep 30, 2025
View reviewed changes
Copy link
Member

@pierrejeambrun pierrejeambrun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just one nit but looks good to me.

We probably need a test for the public API too.

airflow-core/src/airflow/api_fastapi/core_api/openapi/v2-rest-api-generated.yaml
schema:
type: 'null'
title: Response Delete Task Instance
schema: {}
Copy link
Member

@pierrejeambrun pierrejeambrun Sep 30, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure to understand why this was updated.

@github-actions
Copy link

github-actions bot commented Nov 15, 2025

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for your contributions.

@github-actions github-actions bot added the stale Stale PRs per the .github/workflows/stale.yml policy file label Nov 15, 2025
@henry3260
Copy link
Contributor

henry3260 commented Nov 15, 2025

Since this PR has become stale, I would like to help pick it up and get it merged.

@henry3260 henry3260 mentioned this pull request Nov 15, 2025
Merged
@potiuk
Copy link
Member

potiuk commented Nov 15, 2025

Closing this one in favour of #58344

@potiuk potiuk closed this Nov 15, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ashb ashb ashb left review comments

@pierrejeambrun pierrejeambrun pierrejeambrun left review comments

@kaxil kaxil Awaiting requested review from kaxil kaxil is a code owner

@amoghrajesh amoghrajesh Awaiting requested review from amoghrajesh amoghrajesh is a code owner

@ephraimbuddy ephraimbuddy Awaiting requested review from ephraimbuddy ephraimbuddy is a code owner

@rawwar rawwar Awaiting requested review from rawwar rawwar is a code owner

@jason810496 jason810496 Awaiting requested review from jason810496 jason810496 is a code owner

@bugraoz93 bugraoz93 Awaiting requested review from bugraoz93 bugraoz93 is a code owner

@shubhamraj-git shubhamraj-git Awaiting requested review from shubhamraj-git shubhamraj-git is a code owner

Assignees

No one assigned

Labels

area:API Airflow's REST/HTTP API area:task-sdk stale Stale PRs per the .github/workflows/stale.yml policy file

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Execution API / XCom push fails when TaskFlow returns dict with a key containing "/" (slash) in Airflow 3

5 participants

@Brunda10 @henry3260 @potiuk @ashb @pierrejeambrun